Software developers often develop software code in, and for, networking environments where the networked systems have diverse operating systems, or “platforms.” Software development is often performed incrementally. This means the software code is tested for proper performance as smaller incremental portions are completed, prior to incorporating the smaller portion into the whole. This incremental testing methodology contrasts with waiting to test the entire software application as whole, after completion. Software developers utilize small bits of code, or “scripts,” as mechanisms for testing these incremental software portions.
Some testing exercises can be put in simple terms as “breaking the software.” In other terms, it is a process to ensure the quality, correctness and completeness of the software under development. One of the most effective testing methodologies used in contemporary software development and testing environments is “Agile” development. Agile follows a dynamic approach to testing software “builds” gradually, in an iterative fashion. Typically, Agile automatically implements continuous testing as a “test driven” development methodology.
In some circumstances, testing requirements are not stable but dynamically fluctuate with the evolving needs of a customer. Customer satisfaction and return on investment (ROI) are directly related to “product quality” and “cost of quality,” testing therefore has a critical bearing on the success of any software project. It is the responsibility of software testers and developers to generate a product release having minimal defects. One tool in iterative testing is “regression testing,” which is often utilized in each iteration cycle of iterative development effort, such as Agile.
A problem with current regression testing based approaches is that quick and frequent iterations suppress a tester's ability not only to develop and maintain new test scripts, but also complicate identification and selection of every relevant test script from the test bed for execution with respect to the new software release. Because each software release iteration or “version” implies that a new software component is added or an existing software component is modified, a tester must manually re-configure the test scenarios taking care not to miss any test scenarios relating to the new or modified software components.
What is needed is a way to ensure that all the affected components, old and new, along with each of the component “dependencies” are tested thoroughly so that a defect-free product is delivered to the customer.
What is also needed is a way to provide total “black box” test scenario coverage. This is achieved by identifying not only the existing test scripts in the system, but also those missing test scenarios for which test case development is required in order to generate test scripts for all possible underlying component dependencies in a particular release version of the product.